The purpose of this notebook it to provide scientific proof that cats are stranger than dogs (possibly of alien origin). Cats' features are of enormous variety compared to dogs and simply annoying to our brains.

Let's start by following the procedure to rearrange folders: https://github.com/daavoo/kaggle_solutions/blob/master/dogs_vs_cats/01_rearrange_folders.ipynb


In [1]:
from __future__ import print_function
import matplotlib.pyplot as plt
import numpy as np
import os
import sys
import zipfile
from IPython.display import display, Image
from scipy import ndimage
from sklearn.linear_model import LogisticRegression
from six.moves.urllib.request import urlretrieve
from six.moves import cPickle as pickle

from skimage import color, io
from scipy.misc import imresize

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D, Activation, GlobalMaxPooling2D
from keras.layers import merge, Input, Lambda
from keras.callbacks import EarlyStopping
from keras.models import Model
import h5py

np.random.seed(31337)
NAME="ResNet50-300x300-MaxPooling"

# Config the matplotlib backend as plotting inline in IPython
%matplotlib inline


Using TensorFlow backend.

Load original Keras ResNet50 model without the top layer.


In [2]:
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np

resnet_codes_model = ResNet50(input_shape=(300,300,3), include_top=False, weights='imagenet')    
#resnet_codes_model.summary()

Add a Pooling layer at the top to extract the CNN coded (aka bottleneck)


In [3]:
# Final model
model=Model(input=resnet_codes_model.input, output=GlobalMaxPooling2D()(resnet_codes_model.output))

In [4]:
model.summary()


____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_1 (InputLayer)             (None, 300, 300, 3)   0                                            
____________________________________________________________________________________________________
zeropadding2d_1 (ZeroPadding2D)  (None, 306, 306, 3)   0           input_1[0][0]                    
____________________________________________________________________________________________________
conv1 (Convolution2D)            (None, 150, 150, 64)  9472        zeropadding2d_1[0][0]            
____________________________________________________________________________________________________
bn_conv1 (BatchNormalization)    (None, 150, 150, 64)  256         conv1[0][0]                      
____________________________________________________________________________________________________
activation_1 (Activation)        (None, 150, 150, 64)  0           bn_conv1[0][0]                   
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D)    (None, 74, 74, 64)    0           activation_1[0][0]               
____________________________________________________________________________________________________
res2a_branch2a (Convolution2D)   (None, 74, 74, 64)    4160        maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
bn2a_branch2a (BatchNormalizatio (None, 74, 74, 64)    256         res2a_branch2a[0][0]             
____________________________________________________________________________________________________
activation_2 (Activation)        (None, 74, 74, 64)    0           bn2a_branch2a[0][0]              
____________________________________________________________________________________________________
res2a_branch2b (Convolution2D)   (None, 74, 74, 64)    36928       activation_2[0][0]               
____________________________________________________________________________________________________
bn2a_branch2b (BatchNormalizatio (None, 74, 74, 64)    256         res2a_branch2b[0][0]             
____________________________________________________________________________________________________
activation_3 (Activation)        (None, 74, 74, 64)    0           bn2a_branch2b[0][0]              
____________________________________________________________________________________________________
res2a_branch2c (Convolution2D)   (None, 74, 74, 256)   16640       activation_3[0][0]               
____________________________________________________________________________________________________
res2a_branch1 (Convolution2D)    (None, 74, 74, 256)   16640       maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
bn2a_branch2c (BatchNormalizatio (None, 74, 74, 256)   1024        res2a_branch2c[0][0]             
____________________________________________________________________________________________________
bn2a_branch1 (BatchNormalization (None, 74, 74, 256)   1024        res2a_branch1[0][0]              
____________________________________________________________________________________________________
merge_1 (Merge)                  (None, 74, 74, 256)   0           bn2a_branch2c[0][0]              
                                                                   bn2a_branch1[0][0]               
____________________________________________________________________________________________________
activation_4 (Activation)        (None, 74, 74, 256)   0           merge_1[0][0]                    
____________________________________________________________________________________________________
res2b_branch2a (Convolution2D)   (None, 74, 74, 64)    16448       activation_4[0][0]               
____________________________________________________________________________________________________
bn2b_branch2a (BatchNormalizatio (None, 74, 74, 64)    256         res2b_branch2a[0][0]             
____________________________________________________________________________________________________
activation_5 (Activation)        (None, 74, 74, 64)    0           bn2b_branch2a[0][0]              
____________________________________________________________________________________________________
res2b_branch2b (Convolution2D)   (None, 74, 74, 64)    36928       activation_5[0][0]               
____________________________________________________________________________________________________
bn2b_branch2b (BatchNormalizatio (None, 74, 74, 64)    256         res2b_branch2b[0][0]             
____________________________________________________________________________________________________
activation_6 (Activation)        (None, 74, 74, 64)    0           bn2b_branch2b[0][0]              
____________________________________________________________________________________________________
res2b_branch2c (Convolution2D)   (None, 74, 74, 256)   16640       activation_6[0][0]               
____________________________________________________________________________________________________
bn2b_branch2c (BatchNormalizatio (None, 74, 74, 256)   1024        res2b_branch2c[0][0]             
____________________________________________________________________________________________________
merge_2 (Merge)                  (None, 74, 74, 256)   0           bn2b_branch2c[0][0]              
                                                                   activation_4[0][0]               
____________________________________________________________________________________________________
activation_7 (Activation)        (None, 74, 74, 256)   0           merge_2[0][0]                    
____________________________________________________________________________________________________
res2c_branch2a (Convolution2D)   (None, 74, 74, 64)    16448       activation_7[0][0]               
____________________________________________________________________________________________________
bn2c_branch2a (BatchNormalizatio (None, 74, 74, 64)    256         res2c_branch2a[0][0]             
____________________________________________________________________________________________________
activation_8 (Activation)        (None, 74, 74, 64)    0           bn2c_branch2a[0][0]              
____________________________________________________________________________________________________
res2c_branch2b (Convolution2D)   (None, 74, 74, 64)    36928       activation_8[0][0]               
____________________________________________________________________________________________________
bn2c_branch2b (BatchNormalizatio (None, 74, 74, 64)    256         res2c_branch2b[0][0]             
____________________________________________________________________________________________________
activation_9 (Activation)        (None, 74, 74, 64)    0           bn2c_branch2b[0][0]              
____________________________________________________________________________________________________
res2c_branch2c (Convolution2D)   (None, 74, 74, 256)   16640       activation_9[0][0]               
____________________________________________________________________________________________________
bn2c_branch2c (BatchNormalizatio (None, 74, 74, 256)   1024        res2c_branch2c[0][0]             
____________________________________________________________________________________________________
merge_3 (Merge)                  (None, 74, 74, 256)   0           bn2c_branch2c[0][0]              
                                                                   activation_7[0][0]               
____________________________________________________________________________________________________
activation_10 (Activation)       (None, 74, 74, 256)   0           merge_3[0][0]                    
____________________________________________________________________________________________________
res3a_branch2a (Convolution2D)   (None, 37, 37, 128)   32896       activation_10[0][0]              
____________________________________________________________________________________________________
bn3a_branch2a (BatchNormalizatio (None, 37, 37, 128)   512         res3a_branch2a[0][0]             
____________________________________________________________________________________________________
activation_11 (Activation)       (None, 37, 37, 128)   0           bn3a_branch2a[0][0]              
____________________________________________________________________________________________________
res3a_branch2b (Convolution2D)   (None, 37, 37, 128)   147584      activation_11[0][0]              
____________________________________________________________________________________________________
bn3a_branch2b (BatchNormalizatio (None, 37, 37, 128)   512         res3a_branch2b[0][0]             
____________________________________________________________________________________________________
activation_12 (Activation)       (None, 37, 37, 128)   0           bn3a_branch2b[0][0]              
____________________________________________________________________________________________________
res3a_branch2c (Convolution2D)   (None, 37, 37, 512)   66048       activation_12[0][0]              
____________________________________________________________________________________________________
res3a_branch1 (Convolution2D)    (None, 37, 37, 512)   131584      activation_10[0][0]              
____________________________________________________________________________________________________
bn3a_branch2c (BatchNormalizatio (None, 37, 37, 512)   2048        res3a_branch2c[0][0]             
____________________________________________________________________________________________________
bn3a_branch1 (BatchNormalization (None, 37, 37, 512)   2048        res3a_branch1[0][0]              
____________________________________________________________________________________________________
merge_4 (Merge)                  (None, 37, 37, 512)   0           bn3a_branch2c[0][0]              
                                                                   bn3a_branch1[0][0]               
____________________________________________________________________________________________________
activation_13 (Activation)       (None, 37, 37, 512)   0           merge_4[0][0]                    
____________________________________________________________________________________________________
res3b_branch2a (Convolution2D)   (None, 37, 37, 128)   65664       activation_13[0][0]              
____________________________________________________________________________________________________
bn3b_branch2a (BatchNormalizatio (None, 37, 37, 128)   512         res3b_branch2a[0][0]             
____________________________________________________________________________________________________
activation_14 (Activation)       (None, 37, 37, 128)   0           bn3b_branch2a[0][0]              
____________________________________________________________________________________________________
res3b_branch2b (Convolution2D)   (None, 37, 37, 128)   147584      activation_14[0][0]              
____________________________________________________________________________________________________
bn3b_branch2b (BatchNormalizatio (None, 37, 37, 128)   512         res3b_branch2b[0][0]             
____________________________________________________________________________________________________
activation_15 (Activation)       (None, 37, 37, 128)   0           bn3b_branch2b[0][0]              
____________________________________________________________________________________________________
res3b_branch2c (Convolution2D)   (None, 37, 37, 512)   66048       activation_15[0][0]              
____________________________________________________________________________________________________
bn3b_branch2c (BatchNormalizatio (None, 37, 37, 512)   2048        res3b_branch2c[0][0]             
____________________________________________________________________________________________________
merge_5 (Merge)                  (None, 37, 37, 512)   0           bn3b_branch2c[0][0]              
                                                                   activation_13[0][0]              
____________________________________________________________________________________________________
activation_16 (Activation)       (None, 37, 37, 512)   0           merge_5[0][0]                    
____________________________________________________________________________________________________
res3c_branch2a (Convolution2D)   (None, 37, 37, 128)   65664       activation_16[0][0]              
____________________________________________________________________________________________________
bn3c_branch2a (BatchNormalizatio (None, 37, 37, 128)   512         res3c_branch2a[0][0]             
____________________________________________________________________________________________________
activation_17 (Activation)       (None, 37, 37, 128)   0           bn3c_branch2a[0][0]              
____________________________________________________________________________________________________
res3c_branch2b (Convolution2D)   (None, 37, 37, 128)   147584      activation_17[0][0]              
____________________________________________________________________________________________________
bn3c_branch2b (BatchNormalizatio (None, 37, 37, 128)   512         res3c_branch2b[0][0]             
____________________________________________________________________________________________________
activation_18 (Activation)       (None, 37, 37, 128)   0           bn3c_branch2b[0][0]              
____________________________________________________________________________________________________
res3c_branch2c (Convolution2D)   (None, 37, 37, 512)   66048       activation_18[0][0]              
____________________________________________________________________________________________________
bn3c_branch2c (BatchNormalizatio (None, 37, 37, 512)   2048        res3c_branch2c[0][0]             
____________________________________________________________________________________________________
merge_6 (Merge)                  (None, 37, 37, 512)   0           bn3c_branch2c[0][0]              
                                                                   activation_16[0][0]              
____________________________________________________________________________________________________
activation_19 (Activation)       (None, 37, 37, 512)   0           merge_6[0][0]                    
____________________________________________________________________________________________________
res3d_branch2a (Convolution2D)   (None, 37, 37, 128)   65664       activation_19[0][0]              
____________________________________________________________________________________________________
bn3d_branch2a (BatchNormalizatio (None, 37, 37, 128)   512         res3d_branch2a[0][0]             
____________________________________________________________________________________________________
activation_20 (Activation)       (None, 37, 37, 128)   0           bn3d_branch2a[0][0]              
____________________________________________________________________________________________________
res3d_branch2b (Convolution2D)   (None, 37, 37, 128)   147584      activation_20[0][0]              
____________________________________________________________________________________________________
bn3d_branch2b (BatchNormalizatio (None, 37, 37, 128)   512         res3d_branch2b[0][0]             
____________________________________________________________________________________________________
activation_21 (Activation)       (None, 37, 37, 128)   0           bn3d_branch2b[0][0]              
____________________________________________________________________________________________________
res3d_branch2c (Convolution2D)   (None, 37, 37, 512)   66048       activation_21[0][0]              
____________________________________________________________________________________________________
bn3d_branch2c (BatchNormalizatio (None, 37, 37, 512)   2048        res3d_branch2c[0][0]             
____________________________________________________________________________________________________
merge_7 (Merge)                  (None, 37, 37, 512)   0           bn3d_branch2c[0][0]              
                                                                   activation_19[0][0]              
____________________________________________________________________________________________________
activation_22 (Activation)       (None, 37, 37, 512)   0           merge_7[0][0]                    
____________________________________________________________________________________________________
res4a_branch2a (Convolution2D)   (None, 19, 19, 256)   131328      activation_22[0][0]              
____________________________________________________________________________________________________
bn4a_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4a_branch2a[0][0]             
____________________________________________________________________________________________________
activation_23 (Activation)       (None, 19, 19, 256)   0           bn4a_branch2a[0][0]              
____________________________________________________________________________________________________
res4a_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_23[0][0]              
____________________________________________________________________________________________________
bn4a_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4a_branch2b[0][0]             
____________________________________________________________________________________________________
activation_24 (Activation)       (None, 19, 19, 256)   0           bn4a_branch2b[0][0]              
____________________________________________________________________________________________________
res4a_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_24[0][0]              
____________________________________________________________________________________________________
res4a_branch1 (Convolution2D)    (None, 19, 19, 1024)  525312      activation_22[0][0]              
____________________________________________________________________________________________________
bn4a_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4a_branch2c[0][0]             
____________________________________________________________________________________________________
bn4a_branch1 (BatchNormalization (None, 19, 19, 1024)  4096        res4a_branch1[0][0]              
____________________________________________________________________________________________________
merge_8 (Merge)                  (None, 19, 19, 1024)  0           bn4a_branch2c[0][0]              
                                                                   bn4a_branch1[0][0]               
____________________________________________________________________________________________________
activation_25 (Activation)       (None, 19, 19, 1024)  0           merge_8[0][0]                    
____________________________________________________________________________________________________
res4b_branch2a (Convolution2D)   (None, 19, 19, 256)   262400      activation_25[0][0]              
____________________________________________________________________________________________________
bn4b_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4b_branch2a[0][0]             
____________________________________________________________________________________________________
activation_26 (Activation)       (None, 19, 19, 256)   0           bn4b_branch2a[0][0]              
____________________________________________________________________________________________________
res4b_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_26[0][0]              
____________________________________________________________________________________________________
bn4b_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4b_branch2b[0][0]             
____________________________________________________________________________________________________
activation_27 (Activation)       (None, 19, 19, 256)   0           bn4b_branch2b[0][0]              
____________________________________________________________________________________________________
res4b_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_27[0][0]              
____________________________________________________________________________________________________
bn4b_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4b_branch2c[0][0]             
____________________________________________________________________________________________________
merge_9 (Merge)                  (None, 19, 19, 1024)  0           bn4b_branch2c[0][0]              
                                                                   activation_25[0][0]              
____________________________________________________________________________________________________
activation_28 (Activation)       (None, 19, 19, 1024)  0           merge_9[0][0]                    
____________________________________________________________________________________________________
res4c_branch2a (Convolution2D)   (None, 19, 19, 256)   262400      activation_28[0][0]              
____________________________________________________________________________________________________
bn4c_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4c_branch2a[0][0]             
____________________________________________________________________________________________________
activation_29 (Activation)       (None, 19, 19, 256)   0           bn4c_branch2a[0][0]              
____________________________________________________________________________________________________
res4c_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_29[0][0]              
____________________________________________________________________________________________________
bn4c_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4c_branch2b[0][0]             
____________________________________________________________________________________________________
activation_30 (Activation)       (None, 19, 19, 256)   0           bn4c_branch2b[0][0]              
____________________________________________________________________________________________________
res4c_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_30[0][0]              
____________________________________________________________________________________________________
bn4c_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4c_branch2c[0][0]             
____________________________________________________________________________________________________
merge_10 (Merge)                 (None, 19, 19, 1024)  0           bn4c_branch2c[0][0]              
                                                                   activation_28[0][0]              
____________________________________________________________________________________________________
activation_31 (Activation)       (None, 19, 19, 1024)  0           merge_10[0][0]                   
____________________________________________________________________________________________________
res4d_branch2a (Convolution2D)   (None, 19, 19, 256)   262400      activation_31[0][0]              
____________________________________________________________________________________________________
bn4d_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4d_branch2a[0][0]             
____________________________________________________________________________________________________
activation_32 (Activation)       (None, 19, 19, 256)   0           bn4d_branch2a[0][0]              
____________________________________________________________________________________________________
res4d_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_32[0][0]              
____________________________________________________________________________________________________
bn4d_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4d_branch2b[0][0]             
____________________________________________________________________________________________________
activation_33 (Activation)       (None, 19, 19, 256)   0           bn4d_branch2b[0][0]              
____________________________________________________________________________________________________
res4d_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_33[0][0]              
____________________________________________________________________________________________________
bn4d_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4d_branch2c[0][0]             
____________________________________________________________________________________________________
merge_11 (Merge)                 (None, 19, 19, 1024)  0           bn4d_branch2c[0][0]              
                                                                   activation_31[0][0]              
____________________________________________________________________________________________________
activation_34 (Activation)       (None, 19, 19, 1024)  0           merge_11[0][0]                   
____________________________________________________________________________________________________
res4e_branch2a (Convolution2D)   (None, 19, 19, 256)   262400      activation_34[0][0]              
____________________________________________________________________________________________________
bn4e_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4e_branch2a[0][0]             
____________________________________________________________________________________________________
activation_35 (Activation)       (None, 19, 19, 256)   0           bn4e_branch2a[0][0]              
____________________________________________________________________________________________________
res4e_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_35[0][0]              
____________________________________________________________________________________________________
bn4e_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4e_branch2b[0][0]             
____________________________________________________________________________________________________
activation_36 (Activation)       (None, 19, 19, 256)   0           bn4e_branch2b[0][0]              
____________________________________________________________________________________________________
res4e_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_36[0][0]              
____________________________________________________________________________________________________
bn4e_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4e_branch2c[0][0]             
____________________________________________________________________________________________________
merge_12 (Merge)                 (None, 19, 19, 1024)  0           bn4e_branch2c[0][0]              
                                                                   activation_34[0][0]              
____________________________________________________________________________________________________
activation_37 (Activation)       (None, 19, 19, 1024)  0           merge_12[0][0]                   
____________________________________________________________________________________________________
res4f_branch2a (Convolution2D)   (None, 19, 19, 256)   262400      activation_37[0][0]              
____________________________________________________________________________________________________
bn4f_branch2a (BatchNormalizatio (None, 19, 19, 256)   1024        res4f_branch2a[0][0]             
____________________________________________________________________________________________________
activation_38 (Activation)       (None, 19, 19, 256)   0           bn4f_branch2a[0][0]              
____________________________________________________________________________________________________
res4f_branch2b (Convolution2D)   (None, 19, 19, 256)   590080      activation_38[0][0]              
____________________________________________________________________________________________________
bn4f_branch2b (BatchNormalizatio (None, 19, 19, 256)   1024        res4f_branch2b[0][0]             
____________________________________________________________________________________________________
activation_39 (Activation)       (None, 19, 19, 256)   0           bn4f_branch2b[0][0]              
____________________________________________________________________________________________________
res4f_branch2c (Convolution2D)   (None, 19, 19, 1024)  263168      activation_39[0][0]              
____________________________________________________________________________________________________
bn4f_branch2c (BatchNormalizatio (None, 19, 19, 1024)  4096        res4f_branch2c[0][0]             
____________________________________________________________________________________________________
merge_13 (Merge)                 (None, 19, 19, 1024)  0           bn4f_branch2c[0][0]              
                                                                   activation_37[0][0]              
____________________________________________________________________________________________________
activation_40 (Activation)       (None, 19, 19, 1024)  0           merge_13[0][0]                   
____________________________________________________________________________________________________
res5a_branch2a (Convolution2D)   (None, 10, 10, 512)   524800      activation_40[0][0]              
____________________________________________________________________________________________________
bn5a_branch2a (BatchNormalizatio (None, 10, 10, 512)   2048        res5a_branch2a[0][0]             
____________________________________________________________________________________________________
activation_41 (Activation)       (None, 10, 10, 512)   0           bn5a_branch2a[0][0]              
____________________________________________________________________________________________________
res5a_branch2b (Convolution2D)   (None, 10, 10, 512)   2359808     activation_41[0][0]              
____________________________________________________________________________________________________
bn5a_branch2b (BatchNormalizatio (None, 10, 10, 512)   2048        res5a_branch2b[0][0]             
____________________________________________________________________________________________________
activation_42 (Activation)       (None, 10, 10, 512)   0           bn5a_branch2b[0][0]              
____________________________________________________________________________________________________
res5a_branch2c (Convolution2D)   (None, 10, 10, 2048)  1050624     activation_42[0][0]              
____________________________________________________________________________________________________
res5a_branch1 (Convolution2D)    (None, 10, 10, 2048)  2099200     activation_40[0][0]              
____________________________________________________________________________________________________
bn5a_branch2c (BatchNormalizatio (None, 10, 10, 2048)  8192        res5a_branch2c[0][0]             
____________________________________________________________________________________________________
bn5a_branch1 (BatchNormalization (None, 10, 10, 2048)  8192        res5a_branch1[0][0]              
____________________________________________________________________________________________________
merge_14 (Merge)                 (None, 10, 10, 2048)  0           bn5a_branch2c[0][0]              
                                                                   bn5a_branch1[0][0]               
____________________________________________________________________________________________________
activation_43 (Activation)       (None, 10, 10, 2048)  0           merge_14[0][0]                   
____________________________________________________________________________________________________
res5b_branch2a (Convolution2D)   (None, 10, 10, 512)   1049088     activation_43[0][0]              
____________________________________________________________________________________________________
bn5b_branch2a (BatchNormalizatio (None, 10, 10, 512)   2048        res5b_branch2a[0][0]             
____________________________________________________________________________________________________
activation_44 (Activation)       (None, 10, 10, 512)   0           bn5b_branch2a[0][0]              
____________________________________________________________________________________________________
res5b_branch2b (Convolution2D)   (None, 10, 10, 512)   2359808     activation_44[0][0]              
____________________________________________________________________________________________________
bn5b_branch2b (BatchNormalizatio (None, 10, 10, 512)   2048        res5b_branch2b[0][0]             
____________________________________________________________________________________________________
activation_45 (Activation)       (None, 10, 10, 512)   0           bn5b_branch2b[0][0]              
____________________________________________________________________________________________________
res5b_branch2c (Convolution2D)   (None, 10, 10, 2048)  1050624     activation_45[0][0]              
____________________________________________________________________________________________________
bn5b_branch2c (BatchNormalizatio (None, 10, 10, 2048)  8192        res5b_branch2c[0][0]             
____________________________________________________________________________________________________
merge_15 (Merge)                 (None, 10, 10, 2048)  0           bn5b_branch2c[0][0]              
                                                                   activation_43[0][0]              
____________________________________________________________________________________________________
activation_46 (Activation)       (None, 10, 10, 2048)  0           merge_15[0][0]                   
____________________________________________________________________________________________________
res5c_branch2a (Convolution2D)   (None, 10, 10, 512)   1049088     activation_46[0][0]              
____________________________________________________________________________________________________
bn5c_branch2a (BatchNormalizatio (None, 10, 10, 512)   2048        res5c_branch2a[0][0]             
____________________________________________________________________________________________________
activation_47 (Activation)       (None, 10, 10, 512)   0           bn5c_branch2a[0][0]              
____________________________________________________________________________________________________
res5c_branch2b (Convolution2D)   (None, 10, 10, 512)   2359808     activation_47[0][0]              
____________________________________________________________________________________________________
bn5c_branch2b (BatchNormalizatio (None, 10, 10, 512)   2048        res5c_branch2b[0][0]             
____________________________________________________________________________________________________
activation_48 (Activation)       (None, 10, 10, 512)   0           bn5c_branch2b[0][0]              
____________________________________________________________________________________________________
res5c_branch2c (Convolution2D)   (None, 10, 10, 2048)  1050624     activation_48[0][0]              
____________________________________________________________________________________________________
bn5c_branch2c (BatchNormalizatio (None, 10, 10, 2048)  8192        res5c_branch2c[0][0]             
____________________________________________________________________________________________________
merge_16 (Merge)                 (None, 10, 10, 2048)  0           bn5c_branch2c[0][0]              
                                                                   activation_46[0][0]              
____________________________________________________________________________________________________
activation_49 (Activation)       (None, 10, 10, 2048)  0           merge_16[0][0]                   
____________________________________________________________________________________________________
avg_pool (AveragePooling2D)      (None, 1, 1, 2048)    0           activation_49[0][0]              
____________________________________________________________________________________________________
globalmaxpooling2d_1 (GlobalMaxP (None, 2048)          0           avg_pool[0][0]                   
====================================================================================================
Total params: 23,587,712
Trainable params: 23,534,592
Non-trainable params: 53,120
____________________________________________________________________________________________________

The following preprocessing is not proper for the ResNet as it uses mean image rather than mean pixel (I chose VGG paper values) yet it yields little numerical differencies hence works properly and is more than enough for this experiment.

Note that it's required to install github version of keras for preprocessing_function to work with:

pip install git+https://github.com/fchollet/keras.git --upgrade

In [5]:
from keras.preprocessing.image import ImageDataGenerator
def img_to_bgr(im):
    # the following BGR values should be subtracted: [103.939, 116.779, 123.68]. (VGG)
    return (im[:,:,::-1] - np.array([103.939, 116.779, 123.68]))

datagen = ImageDataGenerator(rescale=1., preprocessing_function=img_to_bgr) #(rescale=1./255)

Get the trainign and validation DirectoryIterators


In [6]:
train_batches = datagen.flow_from_directory("train", model.input_shape[1:3], shuffle=False, batch_size=32)
valid_batches = datagen.flow_from_directory("valid", model.input_shape[1:3], shuffle=False, batch_size=32)
test_batches = datagen.flow_from_directory("test", model.input_shape[1:3], shuffle=False, batch_size=32, class_mode=None)


Found 22949 images belonging to 2 classes.
Found 2000 images belonging to 2 classes.
Found 12500 images belonging to 1 classes.

Obtain the CNN codes for all images (it takes ~10 minutes on GTX 1080 GPU)


In [7]:
train_codes = model.predict_generator(train_batches, train_batches.nb_sample)

In [8]:
valid_codes = model.predict_generator(valid_batches, valid_batches.nb_sample)

In [9]:
test_codes = model.predict_generator(test_batches, test_batches.nb_sample)

Save the CNN codes for futher analysys


In [11]:
from keras.utils.np_utils import to_categorical

with h5py.File(NAME+"_codes-train.h5") as hf:
    hf.create_dataset("X_train", data=train_codes)
    hf.create_dataset("X_valid", data=valid_codes)
    hf.create_dataset("Y_train", data=to_categorical(train_batches.classes))
    hf.create_dataset("Y_valid", data=to_categorical(valid_batches.classes))

In [12]:
with h5py.File(NAME+"_codes-test.h5") as hf:
    hf.create_dataset("X_test", data=test_codes)

Compute mean values of codes across all training codes


In [13]:
def get_codes_by_class(X,Y):
    l=len(Y)
    if (len(X)!=l):
        raise Exception("X and Y are of different lengths")
    classes=set(Y)
    
    return [[X[i] for i in xrange(l) if Y[i]==c] for c in classes], classes
    
class_codes, classes=get_codes_by_class(train_codes, train_batches.classes)

cats=np.mean(class_codes[0],0)
dogs=np.mean(class_codes[1],0)
cats=np.abs(cats)
dogs=np.abs(dogs)
# cats=np.log(cats)
# dogs=np.log(dogs)
cats/=cats.max()
dogs/=dogs.max()

Visualize codes as images. As it can be clearly seen, Cats have many different features (plenty of high value - dark spots) while dogs highly activate only two neurons (two distinct dark spots).

It can be concluded that cats activate more brain regions or are more annoying than dogs.

t's even more apparent when looking at the histograms of frequency domain.


In [14]:
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(12, 6))
ax[0,0].imshow(cats.reshape(32,64),cmap="Greys")
ax[0,0].set_title('Cats')

ax[0,1].imshow(dogs.reshape(32,64),cmap="Greys")
ax[0,1].set_title('Dogs')

freq = np.fft.fft2(cats.reshape(32,64))
freq = np.abs(freq)
ax[1,0].hist(np.log(freq).ravel(), bins=100)
ax[1,0].set_title('hist(log(freq))')

freq = np.fft.fft2(dogs.reshape(32,64))
freq = np.abs(freq)
ax[1,1].hist(np.log(freq).ravel(), bins=100)
ax[1,1].set_title('hist(log(freq))')

plt.show()



In [ ]: